Besides the recent impressive results on reinforcement learning (RL), safety is still one of the major research challenges in RL. RL is a machine-learning approach to determine near-optimal policies in Markov decision processes (MDPs). In this paper, we consider the setting where the safety-relevant fragment of the MDP together with a temporal logic safety specification is given and many safety violations can be avoided by planning ahead a short time into the future. We propose an approach for online safety shielding of RL agents. During runtime, the shield analyses the safety of each available action. For any action, the shield computes the maximal probability to not violate the safety specification within the next $k$ steps when executing this action. Based on this probability and a given threshold, the shield decides whether to block an action from the agent. Existing offline shielding approaches compute exhaustively the safety of all state-action combinations ahead of time, resulting in huge computation times and large memory consumption. The intuition behind online shielding is to compute at runtime the set of all states that could be reached in the near future. For each of these states, the safety of all available actions is analysed and used for shielding as soon as one of the considered states is reached. Our approach is well suited for high-level planning problems where the time between decisions can be used for safety computations and it is sustainable for the agent to wait until these computations are finished. For our evaluation, we selected a 2-player version of the classical computer game SNAKE. The game represents a high-level planning problem that requires fast decisions and the multiplayer setting induces a large state space, which is computationally expensive to analyse exhaustively.

Safety is still one of the major research challenges in reinforcement learning (RL). In this paper, we address the problem of how to avoid safety violations of RL agents during exploration in probabilistic and partially unknown environments. Our approach combines automata learning for Markov Decision Processes (MDPs) and shield synthesis in an iterative approach. Initially, the MDP representing the environment is unknown. The agent starts exploring the environment and collects traces. From the collected traces, we passively learn MDPs that abstractly represent the safety-relevant aspects of the environment. Given a learned MDP and a safety specification, we construct a shield. For each state-action pair within a learned MDP, the shield computes exact probabilities on how likely it is that executing the action results in violating the specification from the current state within the next $k$ steps. After the shield is constructed, the shield is used during runtime and blocks any actions that induce a too large risk from the agent. The shielded agent continues to explore the environment and collects new data on the environment. Iteratively, we use the collected data to learn new MDPs with higher accuracy, resulting in turn in shields able to prevent more safety violations. We implemented our approach and present a detailed case study of a Q-learning agent exploring slippery Gridworlds. In our experiments, we show that as the agent explores more and more of the environment during training, the improved learned models lead to shields that are able to prevent many safety violations.

运行时执行是指针对运行时正式规范执行正确行为的理论，技术和工具。在本文中，我们对用于构建AI中执行安全性的混凝土应用程序域的运行时执行器的技术感兴趣。我们讨论了传统上如何在AI领域处理安全性，以及如何通过集成运行时执行器来提供自我学习代理的安全性。我们调查了此类执法者的一系列工作，在该工作中，我们区分了离散和连续动作空间的方法。本文的目的是更好地理解不同执法技术的优势和局限性，重点关注由于AI在AI中的应用而引起的特定挑战。最后，我们为未来的工作提出了一些开放的挑战和途径。

In this paper, we present the Multi-view Extended Videos with Identities (MEVID) dataset for large-scale, video person re-identification (ReID) in the wild. To our knowledge, MEVID represents the most-varied video person ReID dataset, spanning an extensive indoor and outdoor environment across nine unique dates in a 73-day window, various camera viewpoints, and entity clothing changes. Specifically, we label the identities of 158 unique people wearing 598 outfits taken from 8, 092 tracklets, average length of about 590 frames, seen in 33 camera views from the very large-scale MEVA person activities dataset. While other datasets have more unique identities, MEVID emphasizes a richer set of information about each individual, such as: 4 outfits/identity vs. 2 outfits/identity in CCVID, 33 viewpoints across 17 locations vs. 6 in 5 simulated locations for MTA, and 10 million frames vs. 3 million for LS-VID. Being based on the MEVA video dataset, we also inherit data that is intentionally demographically balanced to the continental United States. To accelerate the annotation process, we developed a semi-automatic annotation framework and GUI that combines state-of-the-art real-time models for object detection, pose estimation, person ReID, and multi-object tracking. We evaluate several state-of-the-art methods on MEVID challenge problems and comprehensively quantify their robustness in terms of changes of outfit, scale, and background location. Our quantitative analysis on the realistic, unique aspects of MEVID shows that there are significant remaining challenges in video person ReID and indicates important directions for future research.

Camera images are ubiquitous in machine learning research. They also play a central role in the delivery of important services spanning medicine and environmental surveying. However, the application of machine learning models in these domains has been limited because of robustness concerns. A primary failure mode are performance drops due to differences between the training and deployment data. While there are methods to prospectively validate the robustness of machine learning models to such dataset drifts, existing approaches do not account for explicit models of the primary object of interest: the data. This makes it difficult to create physically faithful drift test cases or to provide specifications of data models that should be avoided when deploying a machine learning model. In this study, we demonstrate how these shortcomings can be overcome by pairing machine learning robustness validation with physical optics. We examine the role raw sensor data and differentiable data models can play in controlling performance risks related to image dataset drift. The findings are distilled into three applications. First, drift synthesis enables the controlled generation of physically faithful drift test cases. The experiments presented here show that the average decrease in model performance is ten to four times less severe than under post-hoc augmentation testing. Second, the gradient connection between task and data models allows for drift forensics that can be used to specify performance-sensitive data models which should be avoided during deployment of a machine learning model. Third, drift adjustment opens up the possibility for processing adjustments in the face of drift. This can lead to speed up and stabilization of classifier training at a margin of up to 20% in validation accuracy. A guide to access the open code and datasets is available at https://github.com/aiaudit-org/raw2logit.

我们开发了一种新类型的模型，以解决通过构建$ \ mathrm {so}^{+}（2,1）$ ecurivariant神经网络来解决多模式光纤的传输效果的任务。该模型利用了已知存在于纤维斑点模式中已知的方位角相关性，并且自然说明了输入和斑点模式之间的空间布置差异。此外，我们使用第二个后处理网络去除圆形伪像，填充间隙并锐化图像，这是由于光纤传输的性质所需的。这种两阶段的方法允许检查由更健壮的身体动机模型产生的预测图像，该模型可能在安全关键的应用程序中或两种模型的输出，从而产生高质量的图像。此外，该模型可以扩展到以前无法实现的成像分辨率，并在256美元\ times 256 $像素图像上显示出来。这是将可训练的参数需求从$ \ MATHCAL {O}（n^4）$提高到$ \ Mathcal {o}（M）$的结果，其中$ n $是像素大小，$ m $是光纤数模式。最后，该模型将在培训数据类别之外的新图像中概括，比以前的模型更好。

我们为交互式数据探索设置中的下一个查询建议提出了一种算法，例如信息收集的知识发现。最先进的查询建议算法基于利用历史交互数据的顺序到序列学习方法。由于学习过程中涉及的监督，这种方法无法适应立即的用户反馈。我们建议使用基于变压器的因果语言模型来查询建议，以适应使用多臂强盗（MAB）框架的直接用户反馈。我们使用来自流行的在线文献发现服务中的日志文件进行大规模的实验研究，并证明我们的算法在基于最先进的变压器的查询建议模型方面大大改善了每轮遗憾，该模型不要使用立即的用户反馈。我们的数据模型和源代码可从https://github.com/shampp/exp3_ss获得

我们开发了一种从2D RGB图像生成3D手网格的旋转等级模型。这保证了当手的输入图像旋转时，所生成的网格经历相应的旋转。此外，这消除了经常通过无旋转标准天例的方法产生的网格中的不希望的变形。通过构建旋转等级模型，通过考虑问题的对称性，我们减少了对非常大的数据集训练的需求，以实现良好的网格重建。编码器在$ \ mathbb {z} ^ {2} $上定义的图像，并将这些映射到组$ c_ {8} $上定义的潜在函数。我们介绍了一种新颖的向量映射函数来将以$ c_ {8} $定义的函数映射到组$ \ mathrm {so}（2）$上定义的潜在点云空间。此外，我们介绍了一种3D投影函数，它从$ \ mathrm {so}（2）$潜空间中学习3D功能。最后，我们使用$ \ mathrm {so}（3）$ arifariant解码器，以确保旋转标准。我们的旋转设备模型优于现实世界数据集的最先进方法，我们证明它可以准确地捕获在输入手的旋转下产生的网格中的形状和姿势。

在这项工作中，我们开发了一种新的方法，名为局部排列的图形神经网络，它为建立在本地节点邻域，通过子图形的构建图形神经网络的框架，同时使用置换等值更新功能。消息传递神经网络的消息被认为是有效应功率的限制，并且最近过度的方法缺乏可扩展性或需要将结构信息被编码为特征空间。这里呈现的一般框架克服了通过通过受限制表示在子图上操作的与全局排列等值相关的可扩展性问题。此外，我们证明了通过使用限制的陈述没有丧失表情。此外，所提出的框架仅需要选择$ k $-hops，用于创建用于为每层使用的子图和选择的表示空间，这使得该方法在一系列基于图形的域中可以容易地适用。我们通过实验验证了一系列图形基准分类任务的方法，在所有基准上展示了最先进的结果或非常竞争力的结果。此外，我们证明使用本地更新函数的使用在全球方法上提供了GPU存储器的显着改进。

我们提出CPT：卷积点变压器 - 一种用于处理3D点云数据的非结构化性质的新型深度学习架构。 CPT是对现有关注的卷曲神经网络以及以前的3D点云处理变压器的改进。由于其在创建基于新颖的基于注意力的点集合嵌入通过制作用于处理动态局部点设定的邻域的卷积投影层的嵌入来实现这一壮举。结果点设置嵌入对输入点的排列是强大的。我们的小说CPT块在网络结构中通过动态图计算获得的本地邻居构建。它是完全可差异的，可以像卷积层一样堆叠，以学习点的全局属性。我们评估我们的模型在ModelNet40，ShapEnet​​部分分割和S3DIS 3D室内场景语义分割数据集等标准基准数据集上，以显示我们的模型可以用作各种点云处理任务的有效骨干，与现有状态相比 - 艺术方法。